Rollout Algorithms for Constrained Dynamic Programming
نویسنده
چکیده
The rollout algorithm is a suboptimal control method for deterministic and stochastic problems that can be solved by dynamic programming. In this short note, we derive an extension of the rollout algorithm that applies to constrained deterministic dynamic programming problems, and relies on a suboptimal policy, called base heuristic. Under suitable assumptions, we show that if the base heuristic produces a feasible solution, the rollout algorithm also produces a feasible solution, whose cost is no worse than the cost corresponding to the base heuristic. 1 Supported by NSF Grant ECS-0218328. 2 Dept. of Electrical Engineering and Computer Science, M.I.T., Cambridge, Mass., 02139.
منابع مشابه
LIDS 2646 Rollout Algorithms for Constrained Dynamic Programming
The rollout algorithm is a suboptimal control method for deterministic and stochastic problems that can be solved by dynamic programming. In this short note, we derive an extension of the rollout algorithm that applies to constrained deterministic dynamic programming problems, and relies on a suboptimal policy, called base heuristic. Under suitable assumptions, we show that if the base heuristi...
متن کاملDynamic Programming and Suboptimal Control: A Survey from ADP to MPC1
We survey some recent research directions within the field of approximate dynamic programming (ADP), with a particular emphasis on rollout algorithms and model predictive control (MPC). We argue that while motivated by different concerns, these two methodologies are closely connected, and the mathematical essence of their desirable properties (cost improvement and stability, respectively) is co...
متن کاملDynamic Programming and Suboptimal Control: A Survey from ADP to MPC
We survey some recent research directions within the field of approximate dynamic programming, with a particular emphasis on rollout algorithms and model predictive control (MPC). We argue that while they are motivated by different concerns, these two methodologies are closely connected, and the mathematical essence of their desirable properties (cost improvement and stability, respectively) is...
متن کاملAverage-Case Performance of Rollout Algorithms for Knapsack Problems
Rollout algorithms have demonstrated excellent performance on a variety of dynamic and discrete optimization problems. Interpreted as an approximate dynamic programming algorithm, a rollout algorithm estimates the value-to-go at each decision stage by simulating future events while following a heuristic policy, referred to as the base policy. While in many cases rollout algorithms are guarantee...
متن کاملNavigation In GPS-Denied Environments Using Approximate Dynamic Programming
Controlling a mobile vehicle to navigate in GPS-denied environments introduces a challenging partially observable control problem with complex constraints. This report presents a combination of various suboptimal control schemes such as open loop feedback control (OLFC), certainty equivalent control (CEC), model predictive control (MPC), and using expected values of estimates as full states to ...
متن کامل